A Simple Oscillatory Short-Term Memory

Oscillatory neural network models have been an increasing focus of study over the last several years. These models consist of recurrent neural networks whose dynamics are characterized by persistent learned/designed rhythmic activity. Here we consider simple oscillatory memories for short-term retention of items occurring as temporal sequences. By incorporating decay as well as interference, we find that it is easy to match behavioral data from human subjects recalling temporal sequences under different situations by adjusting a single parameter in the model. These results suggest that simple oscillatory memories capture at least some key properties of human short-term memory, and might be used effectively in future biologically-inspired cognitive architectures. Modeling Short-Term Memory Using Oscillatory Networks While neural networks using fixed-point attractor states can be effective as memory models and have generated substantial theoretical and experimental analysis, they are typically limited to maintaining only a single pattern at a time in short-term memory. Further, it is difficult to relate activity in these models to neurobiological systems where rhythmic activity, rather than fixed-point attractor states, is the rule (Buzsaki, 2006). In response to these and other concerns, a number of oscillatory memory models have been created and studied during the last several years. In these models, stored/recalled memory patterns are typically represented as rhythmic network activity in which multiple memory patterns are essentially simultaneously active over the same neural substrate. This is possible because the network’s activity oscillates between activity states representing different stored patterns. A remarkably diverse set of oscillatory memory models exists today. Some models are based on theories about the mechanisms underlying theta/gamma activity in specific brain regions such as the hippocampus or neocortex Copyright © 2009, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. (Hasselmo et al, 2002; Ingber, 1995; Koene and Hasselmo 2007, Lisman & Idiart, 1995). Other models that use individual spiking neurons are based on more abstract architectures (Raffone & Wolters, 2001), while still others have adopted a higher-level approach such as WilsonCowan oscillators (Chakravarthy & Ghosh, 1996; Hayashi, 1994; Wang, 1995). Here we are specifically concerned with modeling shortterm memory, the human memory system that retains information over brief time intervals (on the order of seconds) and that has substantial capacity limitations, in contrast to the relatively limitless capacity of more permanent long-term memory (Baddeley, 2000; Cowan, 2001). For example, current evidence suggests that human short-term memory capacity is approximately four items (Cowan et al, 2005). A particularly simple and elegant approach to creating oscillatory short-term memory models is based on minimally modifying Hebbian associative memories having fixed-point attractor states so that they become oscillatory. In the following, we will refer to all such models as simple oscillatory memories. For example, Horn and Usher (1991) produced a simple oscillatory memory by introducing “dynamic thresholds” into Hopfield networks (Hopfield, 1982; Amit, 1989). With this approach, whenever a node has a particular activity level ±1, the threshold of that node gradually changes so that eventually the node switches its activity level to the complementary value. When such a network is presented with an input that is a superposition of multiple stored memories, it is found to oscillate between activity states that represent these individual memories, thereby indicating its recognition/recall of the memories in parallel. Similar behaviors have been produced based upon Hopfield networks modified to use “dynamic synapses” (Pantic et al, 2002) or negative feedback with asymmetric connection weights (Brown & Collins, 2000). Simple oscillatory memories derived from Hopfield networks are intriguing in their simplicity as models of short-term memory. In this paper, we present a simple oscillatory memory based on dynamic thresholds as used by Horn and Usher (1991), except that with our approach 103 Biologically Inspired Cognitive Architectures II: Papers from the AAAI Fall Symposium (FS-09-01)